What is it?
• iPark is a machine learning project that utilises Unity ML-Agents, Reinforcement Learning (RL), and the Unity game engine to create an intelligent system capable of parking a car in various dynamic scenarios.
• It features 2 modes: Training and Evaluation. In Training mode, the agent learns to park the car using Reinforcement Learning techniques, specifically the Proximal Policy Optimisation (PPO) algorithm provided by Unity ML-Agents. In Evaluation mode, the trained model autonomously parks the car in a random empty slot while avoiding collisions.
• I have implemented the entire programming end of the project using Unity Engine and C#, with UI design, AI modelling and refinement, and production being responsibilities of other teammates.
• The project also led to a Research Paper, targeting autonomous car parking.
Motivation
• This project served as the final year undergrad project. It also introduced me to the real-world implementations of machine learning, reinforcement learning, and the Unity ML-Agents toolkit.
• It also serves the purpose of sharpening my team development, leadership, and presentation skills.
Work Done
• Implemented AI agent behaviour, training, evaluation, and environment interaction according to the requirements of the project, alongside the help of the AI engineer in the team.
• Programmed gameplay, UI, and main loop. This made the project more than just an AI tool and increased user interactivity.
• Designed and implemented the training and evaluation levels with appropriate models.
• Created an evaluation system for the models with a data export functionality. This makes comparision and data analysis much easier.
• Acted as a team lead, managing the project timeline, tasks, and team communication.
Code Example
Implemented a custom ML-Agents CollectObservations function to train an AI car agent for parking. The agent receives relative positional, directional, and velocity observations, with shaped rewards encouraging alignment toward the target slot.
public override void CollectObservations(VectorSensor sensor)
{
if (_lastActions != null && _simulationManager.InitComplete)
{
if(_nearestLot == null)
_nearestLot = _simulationManager.GetRandomEmptyParkingSlot();
Vector3 dirToTarget = (_nearestLot.transform.position - transform.position).normalized;
// Add observations (relative position, velocity, direction, orientation)
sensor.AddObservation(transform.position.normalized);
sensor.AddObservation(this.transform.InverseTransformPoint(_nearestLot.transform.position));
sensor.AddObservation(this.transform.InverseTransformVector(_rigidBody.velocity.normalized));
sensor.AddObservation(this.transform.InverseTransformDirection(dirToTarget));
sensor.AddObservation(transform.forward);
sensor.AddObservation(transform.right);
// Reward shaping: encourage moving toward target
float velocityAlignment = Vector3.Dot(dirToTarget, _rigidBody.velocity);
AddReward(0.001f * velocityAlignment);
}
else
{
// Fallback: no valid data
sensor.AddObservation(new float[18]);
}
}
Full Script (GitHub)
Photos
Main Screen:
Evaluation Mode (Choose Model):
Evaluation In Progress:
Model Efficiency Data Export (A data entry is done every time E or the Exit button is pressed):
Training Mode (Only available in Unity Editor with ML-Agents installed):
